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Objectives: The purpose of this study was to test the feasibility of an electronic nursing record system for perinatal care that 
is based on detailed clinical models and clinical practice guidelines in perinatal care. Methods: This study was carried out in 
five phases: 1) generating nursing statements using detailed clinical models; 2) identifying the relevant evidence; 3) linking 
nursing statements with the evidence; 4) developing a prototype electronic nursing record system based on detailed clini- 
cal models and clinical practice guidelines; and 5) evaluating the prototype system. Results: We first generated 799 nursing 
statements describing nursing assessments, diagnoses, interventions, and outcomes using entities, attributes, and value sets of 
detailed clinical models for perinatal care which we developed in a previous study. We then extracted 506 recommendations 
from nine clinical practice guidelines and created sets of nursing statements to be used for nursing documentation by group- 
ing nursing statements according to these recommendations. Finally, we developed and evaluated a prototype electronic 
nursing record system that can provide nurses with recommendations for nursing practice and sets of nursing statements 
based on the recommendations for guiding nursing documentation. Conclusions: The prototype system was found to be 
sufficiently complete, relevant, useful, and applicable in terms of content, and easy to use and useful in terms of system user 
interface. This study has revealed the feasibility of developing such an ENR system. 
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I. Introduction 

The benefits of electronic nursing record (ENR) systems in- 
clude reduced errors, improved quality of care, and lowered 
cost. These benefits can be maximized by using a standard 
nursing terminology, which improves data quality, data shar- 
ing, and decision support. In Korea, an ENR system based 
on standard nursing terminology was introduced for the first 
time at Seoul National University Bundang Hospital in 2003 
[1]. For nursing documentation, nurses at Bundang Hospi- 
tal use nursing statements that were created by combining 
concepts from the International Classification for Nursing 
Practice. The data gathered by this system have been used 
for decision-making and research [2,3] . 

However, the current system has some limitations in ensur- 
ing full semantic interoperability because nursing statements 
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used for documentation have different levels of granularity 
with different qualifiers and value sets. The current ENR sys- 
tem also has not yet incorporated the available evidence on 
clinical practice guidelines (CPGs) and clinical pathways. 

A data model is proposed to ensure the full semantic in- 
teroperability of a terminology-based ENR system. A data 
model expresses the classes of information required and the 
properties of those classes, including attributes, relation- 
ships, and states [4,5]. One such data model is a detailed 
clinical model (DCM) consisting of entity- attribute- value 
triplets. In a DCM, an entity is a core or focus concept of a 
data element. An attribute is a qualifier that represents a data 
entity in more detail. A value set represents a uniquely iden- 
tifiable set of valid concept representations of an attribute. 

There have been several different research projects to devel- 
op data models in health care for a national-level minimum 
data set, electronic health records, a decision support system, 
and data analysis. Examples are IMH Clinical Elements from 
Intermountain Healthcare, ISO 13606 Archetypes, Clinical 
Templates Scotland, Clinical Contents Models, Health Level 7 
templates, and the DCM developed in the Netherlands [4-10]. 

However, the efforts in these approaches have been limited 
to the domain of medical knowledge. While the importance 
of developing a data model for nursing has been addressed 
by nurse informaticists [7], there is only one report on this 
topic in nursing literature by Goossen [11], other than re- 
search results published by the authors of this paper [12- 
16]. These research projects were only pilot studies which 
showed the possibility of model development in a limited 
nursing domain. Nursing statements generated using the en- 
tities, attributes, and value sets of the data models offer new 
possibilities to provide semantic interoperability for nursing 
records. 

Limited use of evidence from research is one of the prob- 
lems encountered in clinical practice. Translating research 
outcomes as evidence for clinical practice is necessary for 
ensuring patient safely and advancing the quality of health- 
care. Evidence-based practice (EBP) is denned as a problem- 
solving approach to practice that involves the conscientious 
use of current best evidence in making decisions about 
patient care in order to promote the selection and imple- 
mentation of the most appropriate intervention [17]. CPG 
with appraised summaries of research and recommendations 
for health care is one strategy with which to use the current 
best evidence [17,18]. As information technology is used to 
improve adherence to guidelines, there were several research 
projects integrating clinical guidelines or pathways into the 
clinical decision support system for pain, pressure ulcers, 
and obesity management in nursing [19-22] . 
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However, these evidence-based clinical decision support 
systems were not linked to an ENR system. Therefore, nurses 
additionally have to document the nursing activities per- 
formed following recommendations. Thus, an ENR system 
linking EBP is proposed to solve these problems. An ENR 
system integrated with EBP not only guides nurses for best 
practice, but also helps them document nursing practice 
around best practice. To our knowledge, there has been no 
previous work on this topic. 

Therefore, an ENR system based on DCMs and CPGs is 
proposed in this study. The purpose of this study is to test 
the feasibility of this system in perinatal care. 

II. Methods 

This study was carried out in five phases: 1) generating 
nursing statements using DCMs; 2) identifying the relevant 
evidence; 3) linking nursing statements with the evidence; 
4) developing a prototype electronic nursing record system 
based on DCMs and CPGs; and 5) evaluating the prototype 
system (Figure 1). 

1. Generating Nursing Statements Using DCMs 

Nursing statements describing nursing assessments, diagno- 
ses, interventions, and outcomes in perinatal care were gen- 
erated using the entities, attributes, and value sets of DCMs 
developed [12,13] by the authors of this paper. 

2. Identifying Evidence 

We searched and reviewed published CPGs related to the as- 
sessment and management of perinatal care. The guidelines 
were searched for in the National Guideline Clearinghouse 
(www.guideline.gov), the Guidelines International Network 
(www.g-i-n.net), the New Zealand Guidelines Group (www. 
nzgg.org.nz), and the National Institute for Health and Clin- 
ical Excellence (NICE; www.nice.org.uk). The search terms 
used included 'labor,' 'pregnancy' 'management of labor,' and 
'delivery care.' 

We extracted recommendations relevant to nursing care 
from these CPGs. These recommendations were rearranged 
by stage of labor and patient problems in each stage. We 
validated this arrangement internally by consulting three 
domain experts who were nurses with more than 10 years of 
clinical nursing experience in maternal care. Each recom- 
mendation was refined through repeated consultation with 
these domain experts. 

3. Linking Nursing Statements with Evidence 

We organized a set of nursing statements around the recom- 
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Figure 1. Development phase of an electronic nursing record system based on detailed clinical models and clinical practice guidelines. 
ICNP: the International Classification for Nursing Practice. 



mendations by referring to literatures such as textbooks on 
maternity and gynecologic care nursing, nursing interven- 
tions classification, and the North American Nursing Diag- 
nosis Association (NANDA), Nursing Outcomes Classifica- 
tion (NOC), and Nursing Intervention Classification (NIC) 
linkage. For example, statements such as 'assess labor pain,' 
'irregular labor pain,' and 'educate breathing, relaxation, and 
visualization techniques' were organized as part of the 'assess 
labor pain' recommendation. 

4. Developing a Prototype ENR System Based on DCMs 
and CPGs 

A prototype ENR system based on DCMs and CPGs was 
developed and implemented on the Microsoft Window 
platform. ASP.NET, CM, WindowsSErver2008, .Net frame- 
works. 5, and Oracle 11 g were used for the design of the 
website and the implementation of the data storage and 
management system. 

The ENR system consists of a terminology server manag- 
ing International Classification for Nursing Practice (ICNP) 
concepts, DCMs, and nursing statements, and a nursing 
documentation system. The terminology server is used not 
only to manage the extended ICNP, the DCMs, and the 
nursing statements, but also the terminology binding of the 
DCM components to the ICNP concepts, and the generation 
of nursing statements. The nursing documentation system 
will be used by nurses to select a recommendation and ap- 
propriate nursing statements under the recommendation to 
create nursing records. Nurses can add values of attributes of 
the statements. 



5. Evaluating the Prototype ENR System 

Twelve domain experts evaluated the prototype ENR sys- 
tem based on DCMs and CPGs. These domain experts were 
nurses with more than five years of clinical nursing experi- 
ence in perinatal care at this hospital. 

Before they evaluated the system, we presented the domain 
experts with nursing statements generated using DCMs, an 
organized set of nursing statements based on the recommen- 
dation, and then we demonstrated the developed system. 
Then, we asked domain experts to rate the completeness, 
relevancy, usefulness, and applicability of the recommenda- 
tions using a questionnaire. In the questionnaire, experts 
were asked whether the recommendations were comprehen- 
sive enough for clinical practice, whether the recommenda- 
tions were relevant enough to clinical practice, whether the 
recommendation were useful enough for expressing clinical 
practice, and whether the recommendations were appli- 
cable enough to clinical practice. The items of the question- 
naire were scored on a five-point rating scale ranging from 
'strongly disagree' to 'strongly agree.' We asked the experts to 
provide comments when they rated items with a score of 1 
or 2. 

Later, we interviewed each domain expert with open-ended 
questions on how easy the system was to use and how useful 
it was. 

III. Results 

1. Generating Nursing Statements Using DCMs 

In total, 799 nursing statements for perinatal care were 
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generated with the entities, attributes, and value sets of 90 
DCMs such as 'labor pain,' 'vaginal discharge,' 'uterine con- 
traction,' and 'fetal movement'. When generating nursing 
statements using DCMs, it is possible to generate nursing 
statements with different levels of granularity, different at- 
tributes and value sets. For example, the 'labor pain' DCM 
has five attributes, including interval, duration, regularity, 
severity, and occurrence. If we choose to generate a nursing 
statement with the duration and interval attributes, 'labor 
pain present (duration, 2 minutes; interval, 3 minutes)' is 
generated. If we choose to generate a nursing statement with 
all the attributes, 'labor pain present (duration, 2 minutes; 
interval, 3 minutes; severity, severe; regularity, regular; oc- 
currence, 20120420 06:50)' is generated. More than 40 nurs- 
ing statements with different levels of granularity can be 
generated using the 'labor pain' DCM, from a very simple 
nursing statement such as 'no labor pain' to a very complex 
statement such as 'labor pain present (duration, 2 minutes; 
interval, 3 minutes; severity, severe; regularity, regular; oc- 
currence, 20120420 06:50)' (Figure 2). 

2. Identifying Evidence 

We identified the following nine CPGs for different stages of 
labor, different types of delivery, and pregnancy risk levels: 
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'antenatal care for normal pregnancy' 'intrapartum care for 
vaginal delivery' 'postpartum care for vaginal delivery' 'in- 
trapartum care for caesarean section (CS),' 'postpartum care 
for CS,' 'pre-eclampsia,' and 'gestational diabetes mellitus' 
(Table 1). We extracted 919 recommendations from these 
nine CPGs. The number of recommendations for 'antenatal 
care for normal pregnancy' 'intrapartum care for vaginal 
delivery' 'postpartum care for vaginal delivery' 'intrapartum 
care for CS,' 'postpartum care for CS,' 'pre-eclampsia,' and 
'gestational diabetes mellitus' was 138, 244, 198, 30, 168, 110, 
and 31, respectively. 

3. Linking Nursing Statements with Evidence 

We extracted 506 unique recommendations related to nurs- 
ing practice from the 919 recommendations and used them 
to create 506 sets of nursing statements. Figure 3 displays 
the structure of the 'intrapartum care for vaginal delivery' 
guideline. For example, 'suspected labor' is a category of 
Stage of Labor and 'labor pain is a category of Patient Prob- 
lem. Under 'labor pain there are recommendations such as 
'assess labor pain,' 'teach breathing technique,' 'encourage 
ambulation,' and explain labor process.' 'Labor pain presents 
at 5-minute intervals' is one of the nursing statements in the 
'assess labor pain' set (Figure 3). 
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Figure 2. Generating nursing statements using the 'labor pain' detailed clinical model. 
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Table 1. Clinical guidelines related to delivery care 
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4. Developing a Prototype ENR System 

These 506 sets of nursing statements were implemented 
in the prototype ENR system based on DCMs and CPGs. 
The following are the linkages between nursing statements 
generated using the DCMs and the recommendations from 
the CPGs. For example, the nursing statement 'labor pain 
present at 5-minute intervals' was generated using the entity, 
attributes, and value sets of the 'labor pain DCM. This nurs- 
ing statement was used as one of the nursing statements for 
the 'assess labor pain recommendation extracted from the 
'intrapartum care for vaginal delivery' CPG. 

Figure 4 shows the components of and relationship be- 
tween the two components of the ENR system. The ter- 
minology server manages the ICNP concepts, DCMs, and 
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nursing statements. The ICNP database (DB) contains ICNP 
concepts. The DCMs DB contains the entity, attribute, value 
set, optionality, data type, and mapping information of these 
entities, attributes, and value concepts to the ICNP concepts. 
The nursing statements DB contains the collection of nurs- 
ing statements generated using the entities, attributes, and 
value sets of DCMs. 

The prototype nursing documentation system displays the 
nursing statements arranged by recommendation for nurses 
to select in nursing documentation. The system developed in 
this study guides not only evidence-based nursing practice 
but also nursing documentation. Figure 5 shows a sample 
screenshot of the nursing records, displaying a list of nursing 
statements arranged by patient problem and recommendation. 
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Figure 4. Elements of the electronic 
nursing record system based 
on detailed clinical models 
and clinical practice guide- 
lines. ICNP: the Internation- 
al Classification for Nursing 
Practice. DCM: detailed clini- 
cal models. 
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5. Evaluating the Prototype ENR System 

The mean scores given for completeness, relevancy, useful- 
ness, and applicability were 4.7, 4.7, 4.8, and 4.8 out of 5.0, 
respectively. Eight recommendations were rated as either 
1 or 2. Five recommendations were rated as irrelevant and 
three recommendations as inapplicable. These recommenda- 
tions were modified and refined. One example rated as ir- 
relevant was the 'perform post-void residual catheterization' 
recommendation in the first stage of labor, because checking 
post-void residual urine is not necessary during the intra- 
partal care of a normal pregnancy. This was modified to 'as- 
sess urine including frequency and characteristics'. Another 
example rated as inapplicable was the recommendation 'in- 
struct patient to report danger signs,' because observing any 
changes in fetal activity during delivery and reporting the 
danger signs to nurses might cause the patients discomfort. 
This was modified to 'instruct caregiver to report danger 
signs' 

The domain experts commented that the system was easy 
to use and useful in displaying sets of nursing statements af- 
ter grouping them according to recommendation by stage of 
labor, type of delivery, and pregnancy risk level, and allow- 
ing nurses to search appropriate statements according to the 
care process. In addition, the new system user interface (UI) 
was not much different from that of the current ENR system. 
However, they pointed out that the statements were unnatu- 
ral and that it took them too much time to select an appro- 
priate value for each attribute. For example, a statement such 
as 'labor pain presents (interval, 5 minutes; duration, 1 min- 
ute)' does not have a natural sentence structure, and nurses 
need to pick values for the interval and duration attributes in 
order to document the labor pain completely. 

IV. Discussion 

This study is one of the first attempts to develop an ENR 
system based on data models and CPGs to support nursing 
practice and nursing documentation. The prototype system 
was found to be sufficiently complete, relevant, useful, and 
applicable in terms of content, and easy to use and useful in 
terms of the system UI. This study has revealed that develop- 
ing such an ENR system is feasible. 

The nursing statements describing patient problems vary 
more than nursing statements describing nursing interven- 
tions and nursing activities by the nursing units and disease 
types. Further, nursing statements describing the same pa- 
tient status and problem have different levels of granularity 
with different qualifiers and value sets [23,24]. For example, 
the presence of pain or the intensity of pain is an impor- 
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tant piece of information in one nursing unit, whereas the 
character of the pain, the quality of the pain, and the clini- 
cal course of the pain are more important in another nurs- 
ing unit. For this reason, it is necessary to provide nursing 
statements with different levels of granularity with different 
qualifiers and value sets to nurses at the point of nursing 
documentation. It was possible for us to generate nursing 
statements with different levels of granularity in the proto- 
type ENR system developed in this study. For example, 41 
different nursing statements were generated using the 'labor 
pain DCM. 

In this study, the nursing statements were generated by 
combining the entity-attribute-value set triplets of DCMs to 
facilitate structured data entry. However, these statements 
were very structured and not natural to human language, 
and it took more time to select an appropriate value from 
a pick-list for each attribute than already pre- coordinated 
statements. To solve this problem, we refined these state- 
ments manually to mimic natural sentences, such as from 
'labor pain (duration, 1 minute; interval, 5 minutes)' to 'labor 
pain presents for 1 minute at 5-minute intervals' Therefore, a 
nursing statement generating system using natural language 
generation technology needs to be developed to automati- 
cally create more user-friendly, readable, and natural nursing 
statements in the future. 

In this study, we used only published CPGs as clinical 
evidence. Although there is no lack of nursing-oriented 
guidelines [17,18], most of these guidelines are focused on 
chronic nursing care or specific nursing problems such as 
pain, pressure ulcers, and breast feeding. Nursing-oriented 
guidelines for acute care are still scarce. In order to develop 
a CPGs-based ENR system, relevant recommendations need 
to be extracted from various guidelines considering different 
target groups and context. 

CPGs are promising tools for closing the gap between 
research and practice, yet the effective and timely imple- 
mentation of evidence into practice remains fragmented 
and inconsistent [25,26] . One barrier is the limited integra- 
tion of guidelines into the care process or the point of care. 
We categorized guidelines by stages of labor, different types 
of delivery, and pregnancy risk levels, and arranged and 
grouped recommendations according to the stages of labor 
and patient problems. Thus, recommendations provided in 
this prototype ENR system were more customized for differ- 
ent stages of labor and patient problems. 

A CPGs-based ENR system will facilitate advanced nursing 
practice and the accumulation of key information gener- 
ated during patient care. Therefore, individual patient data 
gathered with DCMs- and CPGs-based ENR systems will 
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improve data quality, which in turn will provide a founda- 
tion for generating further evidence and measuring clinical 
outcomes. 

However, this study has a few limitations. First, the imple- 
mentation scope of CPGs was limited to perinatal care. 
Second, the evaluation of the ENR system was limited to 
the quality of its content by mainly limited domain experts. 
Therefore, further studies on the CPGs-based ENR system in 
other areas of nursing and evaluation of the system in terms 
of quality improvement in nursing documentation and nurs- 
ing practice are recommended. 
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